eventloop: don't conclude on an unreadable event count - #5
Open
WarpRomo wants to merge 1 commit into
Open
Conversation
Before concluding a bare no-tool turn, the loop checks whether events arrived while the model was generating. That check discarded GetEventCount's error, so a failed read became a count of zero and was indistinguishable from "nothing arrived". Concluding on it is terminal and unrecoverable: db.IsSpine excludes concluded, so RecoverRun skips the session permanently, and an environment notification does not revive a finished session either. Anything queued at the bumped step -- a child_result(crashed), operator input, a background job reporting in -- is lost with it, and the parent is notified with a concluded verdict for work that never finished. Two causes are reachable. On shutdown, session.Registry.CancelAll only ctx-cancels and deliberately leaves DB status alone so RecoverRun can resume; a session in the window between FinalizeStep and this check reads context.Canceled here, and conclude() writes with context.Background(), so the dead context does not stop the terminal write. Separately, a transient store read failure yields the same zero while the following write still succeeds. Route the error through recordFailure, which already classifies both cases: a cancelled ctx returns without writing and leaves the status for recovery, and a db.ErrStore error leaves the session ongoing-but-dead for Recover. Every other store call in the loop already does this; this was the only one that did not, and it sits immediately before the irreversible transition. Both causes are covered by regression tests that fail on the parent commit.
WarpRomo
force-pushed
the
fix/eventloop-conclude-on-unread-event-count
branch
from
September 6, 2026 10:27
9e47228 to
c826171
Compare
WarpRomo
marked this pull request as ready for review
September 6, 2026 10:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The during-generation event count discarded its error, so a failed read was indistinguishable from an empty queue and the agent concluded on it.
Why
An autonomous agent ending a turn with no tool calls checks whether events arrived while the model was generating, then concludes. That check dropped
GetEventCount's error, so a failed read became a count of zero.Concluding is terminal.
db.IsSpineexcludesconcluded, soRecoverRunnever resumes the session and an environment notification cannot revive it. Anything queued at the bumped step is dropped, and the parent gets aconcludedverdict for work that never finished.Shutdown reaches it.
session.Registry.CancelAllonly ctx-cancels and leaves DB status alone soRecoverRuncan resume. A cancel landing betweenFinalizeStepand the check makesGetEventCountreturncontext.Canceled, andconclude()writes withcontext.Background(), so the dead context does not stop the terminal write. A transient store read failure gives the same zero.Implementation
GetEventCount's error instead of discarding itrecordFailure, as every other store call inloopdoesdb.ErrStoreerror leaves the session ongoing-but-dead forRecoverValidation
Interrupted headless runs
amplio headless runagainstdeepseek-v4-flash, interrupted with SIGINT (what Ctrl-C sends) at a swept delay after the model's completion reaches the agent. Identical invocation and identical interrupt schedule on both binaries, so only the patch differs. 27 runs, 3 per delay, final status read from each run's sqlite DB.Below 5 ms the interrupt arrives before the check,
FinalizeStepfails, and both binaries leave the sessionongoing. At 10 ms and above the run has already concluded on its own. At 5 ms the interrupt lands inside the window and the two diverge:An uninterrupted control run on the same setup concludes normally, so the divergence comes from the interrupt rather than the harness.
Tests
Two regression tests, one per cause, fail on
dd403f3and pass with the patch:Focused x50 and
-racex20 clean; full tests, vet, lint and build pass on go1.26.2.The shutdown test forces a real interleaving against the real sqlite store, with nothing stubbed and no error injected. The store-error test does inject its error, noted as such in the file, to cover the
db.ErrStorebranch that the shutdown case does not reach.